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ABSTRACT 


Within federal government contracting, contracting officers are empowered to 
evaluate proposals and determine which contractor should be awarded the contract. With 
multiple variables to consider, managing tradeoffs is an important aspect of the 
evaluation process. As such, there is room for a large amount of subjectivity in the 
evaluation process. That is, one contracting officer may award a particular contractor a 
contract based on his tradeoff analysis, however a different contracting officer, when 
examining the same set of proposals, may have awarded a different contractor the 
contract because his tradeoff analysis was performed differently. There is little 
consistency in the system, especially when proposals for the same contract are similar in 
the variables of interest. Since multiple contracting officers can arrive at different 
conclusions when evaluating the same proposals, there are instances when the wrong 
contractor is awarded a contract, as only one contractor can offer the true best value. 
Thus, the subjectivity in the process needs to be reduced so the contractor offering the 
best value is awarded the contract a high percentage of the time. 

The purpose of this thesis is to examine how the application of existing decision 
support technologies can assist federal government contracting personnel in determining 
which vendor proposal offers the best overall value to the customer in competitive 
solicitations. The intent is to establish a model that, when implemented, will ensure 
contracting evaluate proposals both consistently and fairly. The proposed system 
integrates several decision support technologies. The overall concept is designed using a 
weight-based ranking model, enabled by a multi-criteria decision analysis software 
system. Supporting decision support software packages include an expert system and a 
data warehouse. 

The outcome of this thesis is a proof of concept rather than a fully functional 
decision support application, although certain elements of the overall system will be 
constructed. The intent is to establish how the subjectivity involved in the contract award 
process can be reduced if existing decision support software systems are properly 


integrated in support of a single objective. 
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I. INTRODUCTION 


A. AREA OF RESEARCH 


Within federal government contracting, there are multiple variables for 
contracting officers to consider in determining which contractor is awarded a particular 
contract. These include price, delivery date, and past performance to name a few. The 
contract award decision may not always be obvious, particularly if no contractor’s 
proposal outshines the other proposals in all variables. For example, one contractor may 
offer the lowest price, but has a history of missing deadlines and delivering sub-standard 
products. Thus, managing tradeoffs is an important aspect of the evaluation process. 
That is, one contracting officer may award a particular contractor a contract based on his 
tradeoff analysis, however a different contracting officer, when examining the same set of 
proposals, may award a different contractor the contract because his tradeoff analysis is 
performed differently. There is little consistency in the system, especially when 
proposals for the same contract are similar in the variables of interest. Since multiple 
contracting officers can arrive at different conclusions when evaluating the same 
proposals, there are instances when the wrong contractor may be awarded a contract, as 
only one contractor can offer the true best value to the government. Thus, the 
inconsistency in the process needs to be reduced so the contractor offering the best value 


is awarded the contract every time. 


Currently, this inconsistency is not an issue for contracts awarded under 
simplified acquisition procedures. Within federal government contracting, simplified 
acquisition procedures apply to contracts valued between $2,500 and $100,000. 
Simplified acquisition procedures require only three proposals for each solicitation, as 
opposed to full and open competition required for contracts that exceed $100,000 in 
value. Contracting officers are empowered to evaluate the three proposals and make a 
determination as to which contractor should be awarded the contract based on price 
alone, assuming all three bids meet the technical requirements of the solicitation. This 
avenue for the relaxation of the requirement for full and open competition and the 


1 


simplification of the award deliberation process was created in order to streamline the 
acquisition process for low cost procurements, thereby reducing a portion of the 
administrative cost and burden associated with the federal acquisition system. But since 
the only variable required to be considered is price, there is a strong possibility that the 
contractor offering the true best value is not awarded the contract, even if that 
contractor’s proposal was only marginally more costly than the winning bid in terms of 


price. 


The purpose of this research is to examine how contracts can be consistently 
awarded to the contractor offering the overall best value to the government. Contracts 
awarded under simplified acquisition procedures will serve as the proof of concept, 
although there is strong potential for the principles and recommendations discussed 
herein to be applied to all government acquisition methods. Choosing simplified 
acquisition procedures as the domain for the proof of concept poses an additional 
challenge. That is, since variables in addition to price must be considered in the 
evaluation process, this research will also examine how this can be accomplished without 
contradicting the reforms achieved with the development of simplified acquisition 


procedures. 


The application of existing decision support technologies can assist contracting 
officers in determining which contractor proposal offers the best overall value to the 
government. The intent is to establish a model that, when implemented, will ensure that 
contracting officers evaluate proposals both consistently and fairly. The proposed model 
will integrate several decision support technologies. The overall concept is designed 
using a weight-based ranking model, enabled by a multi-criteria decision analysis 
software system. A supporting decision support software package will be used for the 


expert system. 
B. RESEARCH OBJECTIVES 


The primary research objective is to determine how existing decision support 
system technologies can be integrated and applied to solve business problems in the 


domain of acquisition. Accordingly, a proof of concept is developed to increase the level 
2 


of consistency in evaluating proposals for government contracts under simplified 
acquisition procedures. As a supporting objective, this thesis will explore how this proof 
of concept can be extended and generalized in terms of decision support theories, models, 


and design methodologies required for the subsequent system integration. 


C; RESEARCH DESIGN AND METHODOLOGY 


This research will use the qualitative approach for data collection and analysis and 
will begin with a literature review. The literature review will consist of a review of 
decision support system theory and software systems and federal acquisition regulations. 
Next, the decision support models will be developed. The weight-based ranking model 
and expert system element will be developed based on the outcome of the literature 
review and the author’s knowledge of, and experience in, the Department of Defense’s 
acquisition process. Finally, functional prototypes of both the weight-based ranking 


model and expert system element will be developed. 


D. SCOPE 


The scope of this thesis will focus on developing a model for evaluating proposals 
for government contracts using commercially available decision support software 
systems. Certain individual elements of the model will be constructed; however, other 
elements such as the integration of multiple technologies and the development of a user- 
friendly interface are beyond the scope of this thesis and will only be discussed in 
theoretical terms. Currently, integrating multiple decision support technologies into a 
system with a single user interface requires extensive programming expertise. The 
proposed decision support tool, however, does not require a single user interface to be 
functional. The weight-based ranking model and expert system element can exist as 
separate entities, although a single user interface is ultimately desirable in order to 
minimize human error, decrease the time needed to evaluate proposals, and make the 
entire process more user friendly. Finally, data mining and other decision support 
technologies, although not specifically incorporated into the model, will be discussed in 


terms of their possible contribution to the proposed solution 
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Chapter II of this thesis provides background information on the federal 
acquisition system and decision support system theory. The intent is not to make the 
reader an expert in either area. Rather, Chapter II is designed to give the reader sufficient 
knowledge to comprehend the federal acquisition domain and how decision support 
systems might be applied for positive results. Chapter III shifts focus to the actual model 
that will be employed in the proof of concept, structured around the five basic decision 
support system components. Chapter IV is devoted to presenting the system prototype 
and is organized in the same manner as Chapter III. Finally, Chapter V will offer 


concluding remarks as well as a discussion of future research opportunities. 


I. BACKGROUND 


A. FEDERAL GOVERNMENT ACQUISITION 


The obvious primary objective of the federal government acquisition system is to 
procure and manage the life cycle of the various products and services the government 
needs in order to function. As such, the acquisition workforce is entrusted with billions 
of taxpayer dollars each year, making proper stewardship of government resources an 
equally important objective. While this suggests that the government should always 
award contracts to the bidder offering the lowest price (assuming that bidder is capable of 
fulfilling the requirements) in order to conserve taxpayer dollars, other public policy 
objectives the acquisition system is required to support suggest otherwise. For example, 
part of federal government acquisition policy is to promote small business concerns. 
Thus, even if larger corporations can offer a lower price for a particular contract, if there 
are small businesses capable of fulfilling the requirements, the contract will be set aside 
for a small business. In addition, protecting the environment is a public policy objective 
of the government’s acquisition system. If a product can be supplied cheaper, but 
environmental laws and/or regulations would be violated in the process, the government 
is willing to pay a higher cost. Other acquisition objectives include providing for full and 
open competition (except in the case of set aside programs), procuring commercial 
products and services when possible, rewarding vendors with a history of superior 
performance on government contracts, keeping administrative operating costs at a 
minimum, and conducting business with integrity, fairness, and openness.! The group of 
individuals that ensure these objectives are met for each acquisition is known as the 


Acquisition Team. 


! Federal Acquisition Regulations Part 1.102(b) (U.S. Government Printing Office, Washington, D.C, 
2008). 


1. The Acquisition Team 


The Acquisition Team is comprised of government representatives from the 
procurement, supply, and technical communities. Customer and _ contractor 
representatives augment these individuals as well. Every member of the acquisition team 
is tasked to apply individual initiative and proper business judgment in contributing to the 
acquisition of the best value product or service that meets customer demands in a timely 
fashion. Acquisition team members are not identified all at once. Rather, they are 
identified as their role in the acquisition process becomes relevant. Thus the first 
member of the acquisition team is the customer, who initiates the requirement. As the 
acquisition matures, new members are added to the team, culminating with the contractor 
selected to supply the product or service. Along the way, team members share 
information with each other to ensure the acquisition is the product of a team effort. The 
objective of this approach is to satisfy the customer’s needs in a manner as economical 
and efficient as possible. In order to achieve maximum efficiency, acquisition team 
members are authorized to develop and incorporate innovative strategies and practices in 
performing their duties, provided it is in the best interest of the government to do so and 
the strategies and practices are not in violation of any law, executive order, or other 


regulation.? 
2. Contracting Officer 


One of the most important members of the acquisition team is the Contracting 
Officer. The contracting officer is the only member of the acquisition team authorized to 
enter the government into a contract. Similarly, only a contracting officer can administer 
and/or terminate contracts. A contracting officer gets his authority from a warrant, which 
is a written certification, normally issued by the head of a contracting activity, 
authorizing an individual to enter into contracts on behalf of the government up to a 
designated dollar amount. The dollar amount limitation is established based on the 
individual’s experience, education, knowledge, etc. 


2 Federal Acquisition Regulations Part 1.102(c) (U.S. Government Printing Office, Washington D.C., 
2008). 
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The contracting officer has numerous responsibilities. First and foremost, the 
contracting officer must make sure that the actions required for successful contracting are 
completed, contract terms are adhered to by the government and the contractor, and that 
the government’s interests are protected. Contracting officers are authorized to exercise 
individual business judgment with considerable leeway. It is also the contracting 
officer’s responsibility to ensure that all relevant laws, regulations, and agency policies 
and procedures have been followed before entering into a contract. Other responsibilities 
include ensuring adequate funds are on hand for obligation, ensuring that contractors are 
treated impartially and equitably, and utilizing specialists in other fields (e.g., audit, 


engineering, transportation, law, etc.) as consultants as appropriate.3 
3. Contract Fundamentals 


A contract is a mutually binding legal relationship obligating the seller to furnish 
the supplies or services...and the buyer to pay for them.* In order for a contract to exist, 
there are certain elements that must be present. Naturally, there must be both an offer 
and an acceptance. For example, a contractor “offers” to produce and deliver a product 
to the federal government, who makes the decision to “accept” the offer. In exchange, 
the federal government compensates the contractor for the products rendered. This 
compensation is called “consideration” —another necessary element of every contract. In 
addition, both parties must be legally competent and there must be legality of purpose. 
That is, if the services to be performed or the products to be delivered are illegal, there is 


no contract, regardless if all of the other elements exist. 


Within the federal government, the contracting process begins with procurement 
planning. Procurement planning is the act of determining what to purchase and when to 
purchase it. Once procurement planning is complete, the next step is to plan the 
solicitation. The primary tasks in this stage are to research, document, and refine the 


product or service requirements and to identify possible sources that can fulfill those 





3 Federal Acquisition Regulations Part 1.102-4(a) (U.S. Government Printing Office, Washington, 
D.C., 2008). 


4 Federal Acquisition Regulations Part 2.101 (U.S. Government Printing Office, Washington, D.C., 
2008). 
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requirements. Once the planning is complete, the solicitation is written and publicized. 
The government then obtains bids, offers, quotes, or proposals (depending on the type of 
procurement) in response to the solicitation. Once a specified amount of time has elapsed 
(the exact time is specified in the solicitation), the government proceeds to the Source 
Selection stage, where the proposals are evaluated against the requirements identified in 
the solicitation as well as each other to determine which proposal is chosen as the winner. 
The contract is then awarded to the contractor that submitted that proposal. The contract 
award also ushers in the contract administration stage. This stage involves managing the 
relationship between the government and the contractor throughout the life of the 
contract. The final stage is Contract Close-out, where the contract is completed and 


settled. Any open items are also resolved at this stage. 


4. Acquisition Regulations 


Federal government contracting is governed by several sources. Above all, 
government procurement actions must adhere to all relevant laws and statutes enacted by 
Congress, such as the Contract Disputes Act and the Competition in Contracting Act, two 
of the most significant pieces of legislation related to procurement. The Contract 
Disputes Act established the Board of Contract Appeals through which disputes between 
the contracting officer and the contractor can be resolved. Similarly, the Competition in 
Contracting Act authorized the Comptroller General to hear contractor protests to 
contracting actions and to establish case law relevant to procurement as a product of the 
hearing’s results. Related to Congressional legislation is case law. With respect to 
government contracting, case law emerges as a result of the federal court system hearing 
contract disputes and issuing rulings accordingly. In addition to settling the claims 
between the litigious parties, the rulings also govern future procurement actions of a 


similar nature. 


The actions of the federal government contracting community are also governed 
by the Federal Acquisition Regulation (FAR), a set of policies and procedures applicable 
to every organization within the federal government. Each agency may publish 


regulations in addition to the FAR, provided those regulations do not contradict any 
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element of the FAR. For example, the Department of Defense has created the Defense 
Federal Acquisition Regulation Supplement (DFARS) in order to publish FAR 
implementation guidance and to establish policies exclusive to Department of Defense 
acquisitions. Even agencies within the Department of Defense have published 
supplemental acquisition regulations, such as the Department of the Navy’s Navy 


Acquisition Procedures Supplement (NAPS). 


5; Types of Procurements 


There are three major methods of procurement utilized by the federal government: 
contracting by negotiation, sealed bidding, and simplified acquisition procedures. The 
contracting officer is responsible for choosing which type of procurement is used, using 
guidance provided in the FAR. The FAR, however, does not provide specific 
information on when to use each method. Rather, the contracting officer must fully 
understand the requirements of each solicitation, weigh the pros and cons of each 
procurement method, and make a judgment-based determination on which method to use 


to ensure the adherence to the guiding principles of the FAR. 


a. Sealed Bidding 


Sealed bidding involves evaluating bids on a competitive basis, the public 
opening of bids, and awarding the contract to the bidder offering the lowest price, 
assuming that contractor’s bid is both responsive (fully meets the requirements of the 
solicitation) and responsible (the contractor has the technical and financial ability to 
fulfill the requirements of the solicitation). Sealed bidding is usually employed for the 
purchase of supplies and services that can be specifically described and where 
competition is based only on price and price-related variables. The use of the sealed 
bidding method is contingent on four conditions that must be satisfied. First, there must 
be enough time to allow the solicitation, submission, and evaluation of the sealed bids. 
Secondly, price and price-related variables must serve as the basis on which the contract 
will be awarded. Next, discussions with contractors who submitted bids must not be 


required. Finally, there must be a reasonable expectation that more than one sealed bid 
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will be submitted. In sealed bidding, the solicitation takes the form of an Invitation for 
Bids (IFB). The IFB does not constitute a government offer to procure goods and 
services. Rather, the bids submitted by contractors constitute the offers while the 


government’s contract award serves as the acceptance.° 


Two-step sealed bidding is a special type of sealed bidding, which 
combines negotiation with elements of the traditional sealed bidding method. The intent 
of this method is to realize the advantages of sealed bidding, even when sufficient 
specifications are not on hand. Under two-step sealed bidding procedures, the 
government requests, evaluates, and discusses, as needed, technical proposals from all 
interested contractors. At this point, pricing is not a factor for consideration. The intent 
is to determine which technical proposals are acceptable. After this determination, those 
contractors that submitted acceptable technical proposals submit sealed price bids. After 
publicly opening the bids, the government evaluates them and proceeds to award the 


contract to the lowest bidder, assuming that bidder is both responsive and responsible. 
b. Contracting by Negotiation 


With the contracting by negotiation method, the government and the 
competing contractors exchange information. These information exchanges occur both 
before and after the contractors submit proposals. This method also allows the 
government to award the contract based on criteria other than price. That is, other 
variables such as past performance, technical excellence, management capability and cost 
feasibleness may be incorporated into the source selection criteria. The solicitation takes 
the form of a request for proposal (RFP) in contracting by negotiation. As its name 
implies, the RFP requests interested contractors to submit offers, one of which the 
government may accept and thereby enter into a binding contract with that contractor. 
The government may also choose to enter into negotiations with those contractors whose 
bids fall within the competitive range before choosing which contractor is awarded the 


contract. The competitive range is comprised of the highest rated proposals as measured 


5 Federal Acquisition Regulations Part 14 (U.S. Government Printing Office, Washington, D.C., 
2008). 
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against the evaluation criteria. Before employing the contracting by negotiation method, 
contracting officers must document which of the four conditions required for sealed 


bidding was not met.® 
c. Simplified Acquisition Procedures 


The final major procurement method is Simplified Acquisition Procedures 
(SAP). SAP was established as an effort to streamline the procurement process through 
the reduction of administrative costs. SAP also serves to increase the opportunities for 
small and disadvantaged contractors to be competitive for government contracts. There 
are certain restrictions to using SAP as a procurement method however. The most 
important restriction is that SAP is limited to procurements with an estimated value less 
than or equal to $100,000. But for procurements not expected to exceed that monetary 


threshold, SAP is the preferred method.’ 


Under SAP, solicitations take the form of Requests for Quotations (RFQ). 
As with other types of solicitations, a RFQ is a formal advertisement of a requirement by 
the government. Contractors then respond to the government with a quotation, which 
provides information on price, availability, and other meaningful product information. 
The government evaluates the proposals and then issues an order, or offer, to the 
contractor deemed most qualified to fulfill the requirement. Once the chosen contractor 


accepts the government’s offer, the agreement becomes legally binding. 


SAP can be further broken down into procurements that do not exceed 
$2,500 in value and those that do (i.e., purchases greater than $2,500 and less than or 
equal to $100,000). Purchases less than or equal to $2,500 are called micro-purchases. 
The micro-purchase method further streamlines the administrative cost and burden 
associated with government acquisition through the use of the government-wide 


commercial purchase card or International Merchant Purchase Authorization Card 





6 Federal Acquisition Regulations Part 15 (U.S. Government Printing Office, Washington, D.C., 
2008). 


7 Federal Acquisition Regulations Part 13 (U.S. Government Printing Office, Washington, D.C., 
2008). 
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(IMPAC). Essentially, authorized government personnel purchase items for government 


use using a credit card, thus no solicitation is issued. 
6. Source Selection 


Source selection is the process in which one contractor is chosen to receive the 
contract award. Source selection begins with an evaluation of the proposals that have 
been received in response to a particular solicitation. Source selection goals include 
maximizing competition, minimizing the complexity of solicitation, evaluation, and 
selection decisions, ensuring impartial evaluation, and ensuring selection of the proposal 
with the highest degree of realism. Simplified Acquisition Procedures are an exception to 
the goal of maximizing competition, since contracting officers are required to obtain only 
three bids for each procurement, as opposed to the full and open competition requirement 


for sealed bidding and contracting by negotiation. 


There are numerous evaluation factors that may be considered when selecting a 
source. Obviously, cost is to be considered in every procurement. Other factors may be 
included provided they are relevant to the acquisition, such as past performance, or 
support of public policy objectives. Under Simplified Acquisition Procedures, the 
contracting officer is only required to consider price. As such, contracts awarded using 
SAP are often awarded to the bidder offering the lowest price. Contracting officers are 
not forbidden from considering other evaluation factors when using SAP. In fact, the 
FAR encourages innovative approaches to contracting. Yet most contracting officers 
choose not to do so because it is time consuming to do so and view it as an unduly 


burdensome process, particularly since SAP was established to eliminate such burdens. 


In the federal acquisition system, there are certain circumstances where 
competition is limited in order to support various socio-economic public policy 
objectives. One such set of circumstances is classified as small business “set-asides”. A 
small business set-aside is where a contract is reserved for small businesses, thus 
excluding large businesses from consideration. In general, whenever there are two or 
more small businesses capable of fulfilling a government requirement (not to exceed 


$100,000) at a reasonable price, the contract will be set aside accordingly. As with every 
12 


tule, there are exceptions, however the acquisition objective remains that small 
businesses are to receive any contract where it is determined to be in the interest of 
ensuring that a fair proportion of government contracts for property or services in each 
industrial capacity are placed with small businesses.8 The disadvantaged business set 
aside program is similar in its nature to the small business set-aside program. The 
disadvantaged business program is designed to grant special consideration to businesses 


owned by socially disadvantaged groups such as racial minority groups.? 


Another special consideration impacting full and open competition is the 
Historically Underutilized Business (HUB) Zone program. The HUB Zone program 
seeks to increase government investment/employment in areas of high unemployment 
and underdevelopment. To qualify for this program, the company must be a small 
business, be owned and controlled by United States citizens, have its principal office 
located in the HUB Zone, and have at least 35 percent of its employees residing in the 


HUB Zone.!9 


To summarize, the source selection process can be viewed as a burdensome 
process or a streamlined procedure, depending on the type of procurement, the evaluation 
factors, the nature of the item being purchased, and numerous other considerations. No 
matter the situation, the goal will always be to obtain the best value for the government, 
subject to public policy and acquisition streamlining objectives. Thus, if there are tools 
available that can decrease the complexity of the source selection process without a 
corresponding increase in evaluation and selection time and cost, it makes sense to 
incorporate those tools into the process. A prime candidate for such inclusion is a 


decision support system. 


8 Federal Acquisition Regulations Subpart 19.5 (U.S. Government Printing Office, Washington, D.C., 
2008). 


9 Federal Acquisition Regulations Subpart 19.12 (U.S. Government Printing Office, Washington, D.C., 
2008). 


10 Federal Acquisition Regulations Subpart 19.13 (U.S. Government Printing Office, Washington, 
D.C., 2008). 
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B. DECISION SUPPORT SYSTEM FUNDAMENTALS 


A Decision Support System (DSS) can be defined in numerous ways, depending 
on an individual’s frame of reference and the context in which the term is used. All 
definitions fundamentally convey the same basic idea, that is, a DSS is simply an 
information system that assists in the act of making decisions.!! A DSS can 
accommodate multiple types of inputs and incorporate multiple tools to accomplish its 
mission, ranging from data and documents to optimization models and simulations. 
There are multiple types of DSS and the problem areas in which they can be applied vary 
considerably. For example, a DSS can be applied to a problem as simple as determining 
the optimal driving route from a place of origin to a place of destination given different 
scenarios with respect to variables such as traffic congestion, weather, and fuel 
consumption rates. On the other end of the complexity scale, a DSS that employs 
simulation can be used to predict human behavior for use in complex command and 
control decisions. Regardless of how they are applied, however, all DSS function to 
assist decision-makers in recognizing and resolving problems, completing decision 


process activities, and ultimately arriving at decisions. 
1. Characteristics of a Decision Support System 


In addition to sharing a common purpose, all DSS possess certain principles and 
characteristics. There are three main characteristics common to all DSS. As previously 
mentioned, the first major characteristic of DSS is that they are developed exclusively to 
aid in decision processes. Secondly, DSS are not intended to replace managerial 
judgment. Rather, they are designed to support the decision-maker by providing 
information that can be used in the exercise of managerial judgment. Finally, DSS 
should also be able to react swiftly to the requirements of decision-makers, which can be 
continually dynamic. That is, a DSS should be able to provide managers with the right 


information in the right format at the right time at the right cost. !2 


11 PN. Finlay, Introducing decision support systems (Cambridge: Blackwell, 1994) 3. 


12 DJ. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
<http://dssresources.com/dssbook/>. 
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These three major concepts are augmented by a number of other characteristics. 
For example, a DSS contains a knowledge element that depicts certain elements of the 
decision-maker’s environment, identifies how to complete various activities, and 
distinguishes what conclusions are legitimate in different scenarios. A DSS also has the 
capability to obtain and preserve various types of knowledge, such as procedure keeping, 
rule keeping and descriptive knowledge (through record keeping). In addition, a DSS can 
present knowledge in various forms. That is, knowledge can be customizable and 
provided on an ad hoc basis or it can be presented in a more formalized manner, such as 
through standardized reports.!3 A DSS can provide support to decision-makers in all 
levels of the managerial hierarchy in an organization, to both groups and individuals, and 
to isolated decisions as well as several interdependent and/or sequential decisions. 


Finally, a DSS should be user friendly and easy to build.!4 
Zz Decision Support System Components 


A typical DSS is comprised of five basic components: Data Management, Model 
Management, Knowledge Management, a Graphical User Interface, and the User. The 
interactions between these components are illustrated in Figure 1. While the interactions 
between these five components are useful in illustrating a general DSS framework, not all 
DSS contain all components. For example, a DSS may contain a Model component but 
not a Knowledge component, or vice versa. The data management component of a DSS 
is responsible for managing the numerous tasks required to retrieve, store, and organize 
the meaningful data for the circumstances surrounding the specific decision. The data 
management system’s other responsibilities include managing the system’s security 
features, maintaining and executing data integrity procedures as required, and various 


data administration activities associated with using the DSS. Subsystems such as 





13 Clyde Holsapple and Andrew Whinston, Decision Support Systems: A Knowledge-Based 
Approach (New York: West Group, 1996) 144-5. 








14 Efraim Turban, Decision Support and Expert Systems: Management Support Systems (New York: 
Macmillan, 1990) 110. 
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database(s), a database management system, a data repository, and data query facility 


manage these tasks within the data management component.!5 


Data Model 
Management Management 


Knowledge 
Management 





Figure 1. Decision Support System Components 


The model management component performs functions similar to those of the 
data management component. That is, the model management component carries out 
retrieval, storage, and organizational tasks associated with any mathematical models that 
serve as the DSS’s analytical capability. The model management component is 
comprised of several elements: a model base, model base management system, model 
directory, and model execution, integration, and command. The model base contains the 
actual models that provide the analytical capability. The model base management system 
creates, updates, and changes models, creates and manages reports, and manipulates data 
as needed. The model directory catalogs the entire set of models within a DSS, to include 


model definitions that can be employed by the user when attempting to determine model 





Ip J. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
<http://dssresources.com/dssbook/>. 


16 


availability and capability. Finally, model execution, integration, and command manages 


the actual use of the model and combines models as needed. !® 


The primary function of the knowledge management component is to perform the 
activities associated with managing the problem-solving process. These include tasks 
related to recognizing the problem and developing solutions. The knowledge 
management component synthesizes the data and the models and offers the decision- 


maker an effective application that supports the decision environment in question. 


The design and implementation of the user interface is a critical factor with 
respect to the functionality of the DSS. Users must be able to access and manipulate the 
data, model, and processing components of the DSS without undue difficulty in order for 
the system to offer the required support to the decision environment without interfering 
with the task in question. An effective user interface allows the user to communicate 


with the DSS with relative ease, regardless of the purpose of the communication. 


In order to maximize the effectiveness of the user interface, it is important to 
consider the role and viewpoint of the user. The user plays a critical role in both the 
design and implementation of the system. In considering the user, elements such as skill 
set, motivations, knowledge domain, usage patterns, and organizational roles and how 


they relate to the decision environment should be examined. !7 
3. Types of Decision Support Systems 


There are different approaches to classifying decision support systems, but a 
commonly accepted taxonomy developed by DJ. Power enumerates five major 
categories of DSS: Data-Driven, Model-Driven, Knowledge-Driven, Document-Driven, 


and Communications-Driven. !8 


16 Efraim Turban, Decision Support and Expert Systems: Management Support Systems (New York: 
Macmillan, 1990) 118-23. 





17 George M. Marakas, Decision Support Systems in the Twenty-First Century (Upper Saddle River: 
Prentice-Hall, 1999) 9-11. 


18 DJ. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
<http://dssresources.com/dssbook/>. 
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a. Data-Driven DSS 


Data-Driven DSS stress retrieving and manipulating sizeable databases of 
data. That is, a Data-Driven DSS is frequently a compilation of data, both historical and 
current, from multiple sources that have been stored for trouble-free access and 
examination. Logically, the level of functionality is dependent on the complexity of the 
system. Systems accessed by basic database functions such as query and retrieval offer 
limited, but often inadequate, functionality whereas Data-Driven DSS that employ Online 
Analytical Processing (OLAP) offer a much greater functionality. A Data-Driven DSS is 
capable of providing an organization with both internal data and data concerning the 
organization’s external environment. In addition, the data can be either specific to a 
particular transaction or a summary of multiple transactions. Data-Driven DSS enable 
managers to process data in order to ascertain facts and arrive at conclusions based on the 
patterns or trends they see. The obvious advantages of Data-Driven DSS include the fact 
that they not only supply managers with desired information, but that they do it whenever 
managers need it (and as often as they need it) and in a useable format. Data-Driven DSS 
does have a clear disadvantage in terms of cost. A large investment is often required to 


implement large Data-Driven systems, not to mention updating and maintaining them.!9 
b. Model-Driven DSS 


The second major category of DSS is Model-Driven DSS. As the name 
implies, Model-Driven DSS stress access to and manipulation of one or more models, as 
opposed to databases. As in the case with Data-Driven systems, functionality levels vary 
depending on the nature and complexity of the decision in question. For example, the use 
of simple statistical models offer basic level functionality whereas other systems may 
employ complex models that combine data analysis with sophisticated modeling tools. 
Large databases are typically not required for Model-Driven DSS, as the data and 
parameters needed to run the models are supplied by the decision-maker and are pertinent 
only to the situation in question. Notwithstanding this fact, the ability of Model-Driven 


19 DJ. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
<http://dssresources.com/dssbook/>. 
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DSS to access existing data stores, especially in the form of data warehouses and/or Web- 
resident databases, is a desirable situation, particular when the model requires historical 
data as inputs. Thus, regardless of complexity, mathematical and analytical models 
remain the primary element of a Model-Driven DSS. The models generate output that 
decision-makers can examine and consider before deciding on a particular course of 
action. Every Model-Driven DSS is unique in purpose, precisely designed with a 
particular set of objectives in mind. As such, the models employed are unique as well, 
making the choice of model to use an important fact to consider when planning the 
system. Another characteristic of Model-Driven DSS is the fact that the values of 
important input variables as well as key constraints are dynamic. In fact, they may 
change frequently, depending on the stability of the various environmental and internal 
forces affecting the situation. As with DSS in general, models can take many forms. 
Indeed, modeling techniques can range from simple decision trees and influence 


diagrams to complex simulation and optimization programs.2° 


One of the more common modeling techniques is multi-criteria decision 
analysis. Multi-criteria decision analysis models are often developed using a four-step 
technique known as the Analytical Hierarchy Process (AHP). In comparing the overall 
value of various alternatives available to a decision-maker, AHP considers both 
quantitative and qualitative variables. AHP’s initial task involves diagramming the 
problem in a hierarchal manner. Working from top to bottom, the diagram starts with the 
ultimate objective of the system (e.g., maximize profitability, minimize costs, etc.), 
expands into the relevant variables or selection criteria that must be considered by the 
decision-maker, and ends with the available alternative courses of action. For example, a 
decision-maker seeking to minimize total transportation time when traveling from point 
A to point B might construct a hierarchal representation of the problem similar to the 


diagram shown in Figure 2. 


20 D.J. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
<http://dssresources.com/dssbook/>. 
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Minimize Travel Time 





Figure 2. The Analytical Hierarchy Process 


The next step involves evaluating and comparing the alternatives based on 
available data. The goal of this step is to establish how each alternative compares to 
every other alternative with respect to each variable. The relative importance, or weight, 
of each variable is determined in step three. Finally, step four involves determining the 
ranking of the alternatives based on the results of the previous steps. Essentially, AHP is 
a structured approach to ranking alternatives based on multiple variables and is a useful 
tool in managing tradeoffs. Although the AHP diagram suggests that alternatives can be 
evaluated using simple pairwise comparisons, this approach is rendered less feasible as 
the number of decision alternatives becomes larger. There are several commercially- 
available software systems featuring multi-criteria decision analysis such as Logical 


Decisions for Windows, Expert Choice, and Decision Plus. 


c. Knowledge-Driven DSS 


Knowledge-Driven DSS are interactive computer systems that specialize 
in providing problem-solving capabilities to decision-makers. This problem-solving 
knowledge is drawn from the system’s familiarity and understanding of a specific 
problem domain, comprehension of different types of problems that fall inside that 
domain, and requisite expertise at generating solutions for those problems. All 
Knowledge-Driven DSS share certain characteristics. As stated, the first common trait is 


that all Knowledge-Driven DSS share the same ultimate objective, which is to assist 
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managers in arriving at solutions to problems. Next, this type of DSS relies on data 
structures such as rules, frames, or likelihood information for representing knowledge in 
computer executable form. Finally, the foundation for Knowledge-Driven DSS 
recommendations is human knowledge. That is, Knowledge-Driven DSS do not typically 
think on their own. Rather, they rely on knowledge entered by system designers and 


maintainers in generating recommendations.?! 


As with other DSS categories, Knowledge-Driven DSS can take several 
forms, including World Wide Web sites such as www.wikipedia.com, concepts such as 
Communities of Practice, and expert system technologies. The object of an expert 
system is to mirror the human reasoning process as much as technology will allow. 
Although similar in theory to Model-Driven DSS, expert systems differ in one 
fundamental aspect. That is, Model-Driven DSS revolves around following a series of 
pre-established commands for reacting to a scenario whereas a Knowledge-Driven DSS 
that incorporates an expert system generates a response based on its knowledge base of 
the decision environment and the logical rules that are built into the system to assist in 
solving problems. Additionally, some knowledge/expert systems use inference 
techniques to derive new facts from an existing fact base. In short, whereas Model- 
Driven DSS use a mathematical and/or statistical approach to problem solving, 
Knowledge-Driven DSS relies heavily upon heuristic techniques in arriving at 


recommendations. 


For example, the knowledge base for many expert systems is built using a 
set of rules. Most rules are structured as logical IF-THEN statements, often nested 
several layers deep. Rules formally specify a recommended solution, structured with the 
IF signifying the premise(s) while the THEN portion of the statement constitutes the 
conclusion(s). Thus, the nature of rules revolves around the relationships between 
premise and conclusion as opposed to providing instructions, which is how IF-THEN 


statements are used by many computer programming languages. 


21 DJ. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
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Using rule-based Knowledge-Driven DSS has both advantages and 
limitations. One of the biggest advantages of rules is that they are easy to comprehend. 
Thus, knowledge stored as structured rules allows easy explanations with respect to 
recommendations and the logic the system employed to arrive at those recommendations. 
Also, from the system designer viewpoint, rules are easily adjusted to address changes in 
the decision environment. These advantages of rules are offset to a certain degree by 
several limitations. Namely, rules cannot always be developed to capture knowledge that 
has a high degree of complexity. Additionally, knowledge represented by rules often has 
the tendency to be superficial. Yet despite these limitations, Knowledge-Driven DSS 
designers and builders often favor rule-based systems whenever possible, with the 
understanding that some applications are just too complex to render a rules-based 


approach feasible. 
d. Document-Driven DSS 


Document-Driven DSS is relatively new, and is still evolving. Also 
known as Knowledge Management Systems, a Document-Driven DSS combines multiple 
and diverse storage and processing technological capabilities in order to offer total 
document retrieval and analysis. The World Wide Web is the most commonly 
recognized Document-Driven DSS, in that the web enables access to enormous databases 
of documents, images, audio files, and video files. Common uses of Document-Driven 
databases include accessing and managing policies and procedures, product information, 
catalogs, as well as internal organizational documents of interest such as important 
records, electronic correspondence, and meeting notes. Document-Driven DSS often use 


search engine technology as the primary tool for user interface. 
e. Communications-Driven and Group DSS 


Communications-Driven and Group DSS is the final major category of 


DSS. These DSS stress communications, collaboration, and shared decision-making 


22 DJ. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
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assistance. Examples of this type of DSS include technologies as simple as video tele- 
conferencing, threaded email or web-based community forums. The distinguishing 
feature of a Communications-Driven DSS is that it allows multiple individuals to 
correspond with each other, share information and knowledge, and coordinate their tasks. 
As the name implies, Group DSS enable numerous users to collaborate via software tools 
that are model-driven. This category of DSS is perhaps the fastest growing one in 
concert with the contemporary emphasis on knowledge sharing and knowledge flow 


within organizations.?3 
4. Decision Support System Limitations 


Decision support systems do have certain limitations. For example, DSS are 
currently unable to possess uniquely human decision-making elements such as creativity, 
imaginativeness, or instinct. In addition, DSS are constrained by the computer systems 
upon which they are operating, their design, and the level of knowledge they contain at 
the moment they are run. A third limitation lies in the fact that language and command 
interfaces currently lack the sophistication necessary to enable natural language 
comprehension of user commands (although this is improving). Finally, DSS typically 
are often designed to be limited in scope of application. In turn, this inhibits their utility 
in broader decision-making contexts.2* Also, user acceptance of DSS is sometimes slow 
in forthcoming because of users’ perception of the system challenging their human 


judgment. 
5. Applications in the Government Procurement Domain 


As shown in Figure 3, there are six main stages in the federal government’s 
procurement process: procurement planning, solicitation planning, solicitation, source 


selection, contract administration, and contract close out or termination. Various decision 


23 D.J. Power, Decision Support Systems Hyperbook, Fall 2000, 15 Jan. 2008 
<http://dssresources.com/dssbook/>. 
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concepts and technologies can be employed throughout several stages of the overall 


process to assist contracting officers and other government acquisition professionals. 
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Figure 3. The Government Procurement Process 


For example, both data mining and multi-criteria decision analysis can assist with 
procurement planning. Procurement planning is the process of determining what to 
procure and when to procure it. For example, data mining tools can be used to determine 
what item, among several alternatives, has historically had the smallest rate of failure. 
Similarly, data mining tools can be used to analyze historical relationships between cost 
and quality attributes for previously procured goods and services in preparation for 
current or future procurements. Additionally, in a budget constrained environment, this 
often results in tradeoffs, as the government simply cannot afford to procure everything it 
wants. As such, multi-criteria decision analysis, in concert with expert judgment, can be 
used to manage those tradeoffs. Further, simple decision support tools such as decision 


trees and influence diagrams can be used in this regard as well. 


The solicitation planning stage objectives are to produce the procurement 


documents (e.g., request for proposal, statement of work, etc.) and determine the 
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evaluation criteria if required. There is a prescribed uniform contract format to assist 
acquisition personnel in preparing the procurement documents; however, no similar tool 
exists to assist in determining the evaluation criteria. For this task, an expert system that 
incorporates the collective knowledge of the acquisition workforce can be used to create 


a tool to assist contractors in determining the appropriate evaluation criteria. 


Source selection is the stage in which decision support systems most suitably 
apply. In selecting a contractor, decision support technologies such as multi-criteria 
decision analysis can be used to determine which contractor offers the overall best value 


for the government for a particular procurement action. 


The contract administration phase includes the management of contract changes 
and the monitoring of contractor performance. Contractor performance is a critical 
function, as it may be used to influence future contract award decisions involving that 
contractor. In certain contract types, contractor performance may also impact how much 
profit the contractor earns on that contract. Once again, multi-criteria decision analysis 
can be used to assist acquisition personnel. Contractors can be evaluated on specific 
performance criteria (cost control, schedule management, etc.) in order to make overall 


past performance determinations. 


As the purpose of this thesis is to examine how DSS can be applied to the Source 
Selection stage, the following chapter will focus on a proposed multi-criteria decision 
analysis model. The proposed model will integrate several decision technologies and will 
assist contracting personnel by ranking alternative contractors against multiple weight- 
based variables en route to arriving at a recommendation as to which contractor should be 


awarded the contract. 
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Hl. SOURCE SELECTION SUPPORT SYSTEM MODEL 


This thesis proposes the development of a Source Selection Support System—a 
decision support tool that integrates several commercially available decision support 
software technologies. Specifically, the system integrates a multi-criteria decision 
analysis tool with an expert system and a data warehouse through a common user 
interface. While these components technically need not be integrated in order for the 
overall concept of the system to be realized, developing an integrated decision 
technology environment can provide a richer portfolio of DSS generation capabilities. 
Most standalone DSS focus on providing just one of the basic components (Model 
Management, Knowledge Management, etc.). Applications which require several or all 
of these components are better served by an integrated environment that allows users to 


access various independent components as needed. 


The primary purpose of the Source Selection Support System is to compare three 
separate proposals for government solicitations against established criteria to determine 
the best source in a consistent manner, which in turn can be documented. This decision 
support system is comprised of all five basic DSS components: Model Management, 
Data Management, Knowledge Management, a Graphical User Interface, and the User. 
The user runs the expert system component to determine decision variable weights, as 
influenced by the circumstances of the procurement in question. The user then enters 
contractor and proposal data associated with that solicitation. Scores for each decision 
variable are then calculated for each contractor, at which point a weight-based ranking 
system takes over to ultimately arrive at a recommendation. As the Source Selection 
Support System integrates several commercially available software systems (expert 
system, ranking model, etc.), each of these sub-systems has its own user interface. A 
universal interface is proposed, however, to link all components in order to provide a 
more user-friendly master system. A more thorough discussion of each component 


follows. 
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A. MODEL MANAGEMENT 


The backbone of the Source Selection Support System is a weight-based ranking 
model that follows the principles of the Analytical Hierarchy Process (AHP). As such, 
the model is composed of a series of variables that when weighed accordingly, provide 
the contracting officer with a recommendation as to which contractor offers the best 
overall value to the government for a particular solicitation. The AHP model for the 
Source Selection Support System is shown in Figure 4. Since the ultimate objective of 
the model is to provide the contracting officer with a recommendation, that becomes the 


top tier of the AHP diagram, followed by the variables to be considered and the 


competing contractors serving as the alternatives. 
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Figure 4. The Source Selection Support System AHP diagram 


The independent variables of interest include price, delivery date, warranty, 


customer satisfaction history, on-time delivery history, and report of discrepancy (ROD) 
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history. Other variables may be incorporated into the model, but for the purpose of this 


thesis, the aforementioned six variables will suffice. 


1. Decision Variable 


a. Contractor 


Entry/selection of a particular contractor and respective bid initiates the 
model. Thus, selection of an alternative constitutes a decision variable. Contractors that 
conduct business with the federal government are uniquely identified by a five digit code 
known as the cage code. If the contractor is already recorded in the database, the cage 
code is used to retrieve that contractor’s relevant information. If the contractor is not yet 


in the database, there is an option to insert the contractor. 


2. Independent Variables 


a. Qualified Disadvantaged Business 


The federal government offers special consideration to businesses owned 
by women, racial minorities, and United States military veterans. Government 
contracting officers are authorized to award contracts to businesses that qualify for this 
program even if it results in an increase in cost to the government, provided the 
business’s net worth does not exceed $750,000. The intent is to satisfy public policy 
objectives by awarding government contracts to socially disadvantaged businesses that 
otherwise may not be able to realistically compete for government work. Despite this 
objective, contracting officers will not award a contract to a disadvantaged contractor if 
the price difference is too great. Thus, the Source Selection Support System applies a 
percentage-based price adjustment on the quoted price from disadvantaged businesses in 
order to account for the price difference limitation. For example, the contracting officer 
may use ten percent as the acceptable difference between a disadvantaged contractor’s 
quoted price and a non-disadvantaged contractor’s quoted price. The price adjustment 
percentage can change, depending on factors such as individual contracting activity 


policy, the nature of the procurement, etc. 
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The model gets disadvantaged business status (yes or no) for each 
contractor from the data warehouse. As for the value of the percentage-based price 
adjustment, this is directly entered by the user, as it could vary depending on command 
policy and the characteristics of the procurement. If a contractor does not qualify for this 
price adjustment, its quoted price is unchanged. If a contractor does qualify for this price 


adjustment, its quoted price is reduced by the following amount (PApjs): 


PApys = P, * (AP pis) 


Where: 
PA); = Disadvantaged business price adjustment 
P= Initial Proposal Price 
AP,,; = Disadvantaged business price adjustment percentage 
An example disadvantaged business price adjustment calculation is shown 
in Table 1. 














Price Adjustment (APp;s) = 10 % 

Company P; Disadvantaged Business? P, * (APpis) Revised Price 
A $1,000 Yes $1,000 * 10% = $100 $900 
B $950 No N/A $950 
C $975 No N/A $975 














Table 1. | Example Disadvantaged Business Price Adjustment Calculation 


The price adjustment is applied to all qualifying contractors, regardless of 
how much difference there is between prices. If the revised price is still higher than the 
non-qualifying contractors, the proposal from the disadvantaged contractor will still be 
considered, as the remaining variables used by the model may still result in the 


disadvantaged contractor offering the best overall value despite the higher price. 
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b. Qualified HUB Zone 


In an effort to spread federal government work across a wider geographic 
area and support businesses in local economies that may be suffering, the government 
gives special consideration to contractors located in Historically Underutilized Business 
(HUB) Zones. In order to qualify for HUB Zone status, the business must be owned and 
controlled by United States citizens, have its principal office physically located in the 
HUB Zone, and have a minimum of 35 percent of its employees residing in the HUB 
Zone. Once again, a percentage-based price adjustment is applied to HUB Zone 
contractors bidding on a solicitation. Similar to the Disadvantaged Business variable, the 
model obtains a contractor’s HUB Zone status (yes or no) from the data warehouse. The 
value of the percentage-based price adjustment is directly entered by the user, as it could 
vary for the same reasons as the Disadvantaged Business variable. Once again, if a 
contractor does not qualify for this price adjustment its quoted price is unchanged. If a 
contractor does qualify for this price adjustment, its quoted price is reduced by the 


following amount (PA qypz): 


PAnyg = P, * (APuyp ) 


Where: 

PAyyz = HUB Zone price adjustment ($) 

P= Initial Proposal Price 

AP; = HUB Zone price adjustment percentage 

An example HUB Zone business price adjustment calculation is shown in 
Table 2. 
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Price Adjustment (APyusg) = 10 % 
Company P; Qualified HUB Zone? P, * (AP yup) Revised Price 
A $1,000 Yes $1,000 * 10% = $100 $900 
B $950 Yes $950 * 10% = $95 $855 
C $975 No N/A $975 

















Table 2. | Example HUB Zone business price adjustment 


The rules for application of the Disadvantaged Business price adjustment 
apply for this variable as well. That is, the HUB Zone price adjustment is applied to all 
qualifying contractors, regardless of how much difference there is between price. If the 
revised price is still higher than the non-qualifying contractors, the proposal from the 
HUB Zone contractor will still be considered, as the remaining variables used by the 
model may still result in the HUB Zone contractor offering the best overall value despite 


the higher price. 
Cc. Delivery Date Score 


This variable rewards the contractor offering the earliest promised 
delivery date. The contractor with the earliest promised delivery date as indicated on the 
proposals receives a score of 100 percent. The other two contractors receive scores 
proportionate to the deviation (in days) of their delivery dates from the earliest delivery 
date. The model gets delivery date information via direct data entry, with the information 


source being the contractor’s proposal. The Delivery Date Score is calculated as follows: 





Spp = Delivery Date score 
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Dow = The number of days from expected contract award date to the 


earliest promised delivery date 


D.xyp = The number of days from expected contract award date to this 


individual contractor’s promised delivery date 


An example delivery date score calculation is shown in Table 3. 





Expected Contract Award Date: 7/1/2008 



































D,,, —D 

Company | Delivery Date Dino 1 2A Spp 
Drow 

A 8/01/2008 sil 1, since 31 = Drow 100 % 

B 9/15/2008 76 1-[(76-31)/31]= -0.4516 - 45 % 

C 8/14/2008 44 1-[(44-31)/31]= 0.5806 58 % 





Table 3. | Example delivery date score calculation 


Note that the calculation can result in a negative score. Due to a software 
limitation that does not allow negative values for scores, negative scores must be reset to 


zero. Thus, the Delivery Date Score for Contractor B is zero percent. 


d. Warranty Score 


The warranty score is calculated based on the length (in months) of the 
warranty. The company with the warranty covering the longest period is assessed a score 
of 100 percent. The other two companies are assessed scores proportionate to the 
deviation of the lengths of their warranties to the length of the warranty covering the 
longest period. The model gets warranty information via direct data entry, with the 
information source being the contractor’s proposal. The warranty score is calculated as 


follows: 
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Wircu 
Where: 
Sy, = Warranty score 
Wincy = The number of months of coverage the longest warranty offers 
W inp = The number of months of warranty coverage offered by this 
individual contractor 
An example warranty score calculation is shown in Table 4. 
Wino —W, 
Company Wino 1+ [Mae Mos Sw 
Wircn 
A 0 (no warranty) 1+[(0-48)/48] = 0 0 % 
B 42 1+[(42-48)/48] = 0.875 87.5 % 
‘s 48 1+[(48-48)/48] = 1 100 % 
Table 4. Example warranty score calculation 


e. Customer Satisfaction Score 


The value of this variable is the average percentage score for the 
contractor on a uniform customer satisfaction survey. The survey is issued to customers 
of this contractor on government contracts, with the data recorded in the data warehouse. 
The survey uses a likert scale, enabling customers to evaluate contractors on various 
criteria using a numerical scale from 0 to 10. The Customer Satisfaction Score is 


calculated as follows: 
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(dS 
S¢s =| =— |for all Sis 
n 
Sc; = Customer Satisfaction score 
Sis; = The average score for each individual survey filled out for this 


contractor 
n= The number of contracts for this contractor for which a customer 


satisfaction survey has been submitted 


An example customer satisfaction score calculation is shown in Table 5. 





Contractor A 
Contract Number Average Individual Customer Satisfaction Score 
N38259-06-C-5839 82 % 
N86938-07-D-2358 91 % 
N38259-07-D-3321 98 % 
Overall Customer Satisfaction Score = (82 + 91 + 98)/3 = 90.33 


























Table 5. | Example customer satisfaction score calculation 


f On-time Delivery Percentage 


This variable measures the percentage of government contracts the 


contractor has won where the promised delivery date was met. The intent of this variable 


is to penalize contractors who have failed to meet the delivery terms of their contracts. 


The more missed delivery dates a contractor has on its record, the lower the on-time 


delivery percentage. The On-Time Delivery Percentage score is fed data from the data 


warehouse and is calculated as follows: 


35 


Sor = On-time Delivery Percentage score 
n = The number of government contracts this contractor has been 


awarded 


Nor = The number of government contracts this contractor has fulfilled 


on time. 
g. Report of Discrepancy (ROD) Percentage 


Reports of Discrepancy are complaints against a contractor, filed by a 
customer, on a federal government contract. They can be either as a result of the wrong 
product received or the wrong service performed or as a result of poor product/service 
quality. A contractor’s score for this variable is the percentage of federal government 
contracts the contractor has been awarded for which no ROD was filed. ROD data is 


obtained from the data warehouse with the ROD percentage score calculated as follows: 


N 
Srop -1-( 2 | 
n 


Where: 


Srop = ROD percentage score 
n = The number of government contracts this contractor has been 


awarded 


Nrop = The number of government contracts this contractor has been 


awarded for which a ROD was submitted. 
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An example ROD percentage score calculation is shown in Table 6. 
































Company n Nrop Srop 

A 211 4 98 % 

B a 11 81% 

C 163 6 96 % 
Table 6. Example ROD percentage score calculation 


3. Dependent Variables 


a. Price Score 


A contractor’s price score begins with the contractor’s revised bid. The 
revised bid is the quoted price after applying any appropriate price adjustments (e.g., 
disadvantaged business). The contractor that offers the lowest price after the price 
adjustments are applied is awarded a score of 100 percent. The contractors who do not 
offer the lowest price after price adjustments are applied are assigned scores 
proportionate to the deviation between their revised bids and the lowest revised bid. This 
variable gets its information from the independent variables Price, Qualified 
Disadvantaged Business, and Qualified HUB Zone. The Price Score is calculated as 


follows: 





th (S: ae 
P 


RB LOW 
Where: 
Sp= Price score 
B,= Initial bid amount 


RBiow = Lowest revised bid (1.e., after application of price adjustments) 


AP = Total price adjustment 
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An example price score calculation is shown in Table 7. 





Sp 


B,—AP)-RB 
Company |B; AP |B, -AP 1-[ SAF | 


RB ow 





A $1,100 | $0 $1,100 | 1-[(1,100-1,035)/1,035] = 0.9372 | 93.72 % 





B $1,150 | $115 | $1,035 1, since $1,035 = RBrow 100 % 











C $1,250 | $125 | $1,125 | 1-[(1,125-1,035)/1,035] =0.913 | 91.3 % 




















Table 7. Example price score calculation 


b. Overall Acceptance Score 


The sum of the scores of all other variables, multiplied by their respective 
weights. The value of this variable for a contractor is compared to the value for the other 
contractors that submitted proposals. Whichever contractor achieves the highest score for 
this variable is recommended for contract award. The overall acceptance score is 


calculated as follows: 


Stor = (W, = Sp) a (Wy i Sy) a5 (Wop i Sp) a (Wes - Sos) a (Wor = Sor) + (Wrop si Srov) 


Where: 
S;or = Overall acceptance score for a particular contractor’s bid 
Wx= _ Percentage weight assigned to variable X 
4. Influence Diagram 


The influence diagram goes into effect once the three bids required under 


simplified acquisition procedures are in hand. The influence diagram shown in Figure 5 
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depicts how the Source Selection Support System model is structured. The model is 
applied three times for each contract, since it applies separately to each contractor bid 


being evaluated. 
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Figure 5. Source Selection Support System influence diagram 
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5. Constraints 


Most of the constraints for this model are addressed in the request for proposal. 
For example, a variable such as delivery date would naturally have some kind of 
constraint. Realistically, customers are only going to wait a certain amount of time to 
receive the product/service called for in the contract. Yet in theory the model appears to 
allow for a contractor to have what the customer might consider an unreasonable amount 
of time to pass from contract award to delivery date and still achieve the highest overall 
acceptance score, depending on the weights assigned to each variable. The proceeding 


example illustrates this possibility. 


Three proposals are received for a government contract. The delivery date score 


is calculated in Table 8 (note that the lower limit for delivery date score is zero). 



































Expected Contract Award Date: 7/1/2008 
Contractor | Delivery Date Dinp 1 [Pes Pow Spp 
Drow 
A 8/01/2009 397 1, since 397 = Drow 100 % 
B 8/13/2009 409 1-[(409-397)/397]= 0.9698 97 % 
C 8/14/2012 1,505 1-[(1,505-397)/397]= -1.7909 0% 


Table 8. _ Delivery date scores for example scenario 


The scores on all variables for each contractor are listed in the Table 9. 
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Contractor A Contractor B Contractor C 
Variable Weight | Score | Adjusted | Score | Adjusted | Score | Adjusted 
Price Score | 40% | 33% | 13.2% | 37% | 148% | 100% | 40% 
Delivery Date | 494 | 190% | 20% | 97% | 194% | 0% 0% 
Score 
ay 10% | 80% 8 % 80 % 8 % 100% | 10% 
Score 
US OMer 10% | 90% 9% 92% | 92% | 95% | 9.5% 
Satisfaction 
re 10% | 100%} 10% |100%| 10% |100%| 10% 
Delivery % 
ROD 
10% | 100% 10% 97% | 97% | 92% | 92% 
Percentage 
Overall 10.2 % T1.1 % 18.7 % 
Acceptance 














Table 9. | Example scenario variable scores for all contractors 


Despite a delivery date three years after the other two contractors, Contractor C 
wins the contract because it is much better on the Price Score variable, which is weighted 
two times as heavily as Delivery Date Score and four times as heavily as any other 
variable. Contractor C’s delivery date, however, may be outside the realm of 
reasonableness for the customer. In this situation, a constraint seems to be necessary. 
Fortunately, the model does not need to account for constraints like this because the 
request for proposal identifies the constraints long before the model is ever applied. In 
other words, if a contractor’s proposal does not satisfy the constraints in the request for 
proposal, such as required delivery date, the proposal is thrown out before applying the 
model. There are no variables to which a constraint logically applies where that 


constraint cannot be incorporated into the request for proposal. 
B. KNOWLEDGE MANAGEMENT 


Even when the competing proposals are in hand, there is still a critical knowledge 
component that must be in place before the model can be applied. Recall that the model 
follows the principles of the analytical hierarchy process. That is, it employs weight- 
based ranking to arrive at its recommendation. As such, the model is still missing the 


weight for each variable. 


4] 





Determining the appropriate amount for the variable weights is not a simple task. 
One possible approach to this challenge is to assemble a team of experienced subject 
matter experts within the individual contracting activity who can collectively determine 
the appropriate relative weights for each variable. Realistically there is no single weight 
distribution plan that is appropriate to every scenario a contracting officer is likely to 
face. Consider the following two situations. In situation one, the customer is running 
low on funds due to other necessary purchases. This customer will be able to afford the 
product being procured under contract, but would like to do so at the lowest cost possible 
(assuming the product satisfies required performance parameters). Furthermore, the 
customer does not require the product any sooner than the required delivery date 
indicated on the request for proposal. In this situation, it would be appropriate for the 
contracting officer to weigh the price score variable more heavily than usual and weigh 
the remaining variables less in order to compensate. As such, the contracting officer may 


elect to use a set of weights similar to those listed in Table 10. 











Variable Weight 
Price Score 70 % 
Delivery Date Score 10 % 
Warranty Score 5 % 
Customer Satisfaction Score 5 % 
On-Time Delivery Percentage 5 % 
ROD Percentage 5 % 











Table 10. Sample price-intensive variable weight-distribution plan 


Alternatively, situation two involves a customer requesting the procurement of a 
product critical to a primary mission area. Although there is a required delivery date 
indicated on the request for proposal, since the customer is deploying in several weeks, 
the earlier the item is delivered the better. An earlier delivery will allow more time for 
contractor technical support should onsite training be required. As such, the customer is 


willing to pay a premium if it means getting the product sooner. Clearly in this situation 
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delivery date should be weighed heavier than it is under normal circumstances. The 


contracting officer may elect to use a set of weights similar to those listed in Table 11. 








Variable Weight 
Price Score 50 % 
Delivery Date Score 30 % 
Warranty Score 5 % 
Customer Satisfaction Score 5 % 
On-Time Delivery Percentage 5 % 
ROD Percentage 5 % 











Table 11. Sample delivery date-intensive variable weight-distribution plan 


Note that just because a particular variable’s relative importance increases, it does 
not necessarily mean that it becomes the most heavily weighted variable. Logically, 
price will always be the most important variable because a contractor’s performance on 
every other variable will always be ultimately acceptable. Otherwise, the contractor 
would have been suspended (or debarred entirely) from federal government work or have 
its proposal rejected. For example, if a contractor’s ROD percentage score is so low that 
it is unacceptable, than that contractor would not be permitted to continue bidding on 
government work (i.e., suspension or debarment). Similarly, if a contractor’s delivery 
date is unacceptable (i.e., it does not satisfy the required delivery date indicated on the 
request for proposal), its proposal will be rejected long before the model is applied. Price 
Score is the only variable where there is no constraint that prohibits it from being 
considered. That is, the model accepts proposals from contractors offering a 
comparatively low price while at the same time accepting proposals from contractors 
offering a much higher price. Since the price range among the alternative proposals may 
vary widely, and since every contractor is technically acceptable with respect to the other 


variables, price score should always be the primary discriminator. 


It therefore becomes necessary to develop a series of likely scenarios a 
contracting officer is likely to encounter and determine a specific weight distribution plan 
for each of those scenarios. A simple decision tree can be used to model the scenarios 


43 


and record the weights for each variable within those scenarios. Figure 6 depicts a 
portion of one such decision tree. Although this sample decision tree only includes three 


variables, it can be expanded in order to accommodate other variables. 














Decision , Price ' Delivery Date \ Warranty 
Variable , Importance , Importance \ Importance 

| —s[_ Very 
: | of asap}! sl Saneatat | 
I I I 
1 1 , 
1 l | —>_ Vey | 
| —+[ High} + Farty _} + + Somewhat | 
I | | 
; ; ; 
l I |\-—__Vey | 
[Somewhat | 
1 I l ~ 
; ; 
I ! | [Vey _| 
| _.t—asap_}'| 4 Somewhat | 
I l l 
1 1 i 
l ! lof Vey | 

| Contractor | | Medium | | Early | | Somewhat | 
; 
1 1 I | Very | 
. | Somewhat _| 
, , | Little 
| + Vey 1 
. Asap}! | +( Somewhat | 
l 1 I 
1 i , 
1 1 | +L Vey 
I I I 

[Low [Early | [ Somewhat | 
1 Low 1 1 
i , ; 
1 i |-_ Vey 
[ Somewhat | 
, 8 Little 
Figure 6. Portion of sample decision tree 
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Note that the user must select one of four price ranges within which the contract is 
expected to fall. The lower the expected price range, the lower the weight for this 
variable will be. Conversely, if the contract is expected to cost the customer a high dollar 
amount, the weight for this variable will be higher. This is due to the budget limitations 
most, if not all, customers face. The more money spent on the contract, the less money 
available for other purchases. Thus, as the contract value increases and requires more 
financial resources from the customer, minimizing costs becomes even more important 
due to the need to fund other requirements. Once the user determines in what range the 
contracts expected price will fall, he then proceeds to the next variable, Delivery Date. In 


this decision tree, there are three scenarios for Delivery Date: 


1. If the item being procured is a mission critical item, the weight for 


Delivery Date score is increased. 


2. If the item being procured is not mission critical, but is requested by the 
customer to be delivered as soon as possible, the weight is increased above 
normal levels, but not to the point where it matches the Delivery Date 


score weight for a mission critical item. 


3. If there is not a compelling need for the product and the customer can wait 
until the required delivery date (as indicated on the request for proposal) to 
receive the item, the weight for Delivery Date score is comparatively 


lower than the weight under the other two scenarios. 


After the weight for Delivery Date score is determined, the system performs 


similar functions for the remaining variables. 


The problem still remains, however, as to the best way to determine the 
appropriate weight for each variable in each scenario. Unlike most situations in 
government work, there is no statute or regulation that prescribes either the answer or the 
way in which to arrive at the answer. Fortunately, contracting personnel are uniquely 
qualified to develop a reasonable solution due to the need for them to exercise judgment 
in managing tradeoffs and their ability to rely on experience when awarding contracts. 


Accordingly, the most effective way to determine the proper weight distribution plans for 
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each scenario (after modeling the scenarios using a decision tree) may still be to assemble 
a team of experienced contracting personnel at each activity, who in turn can reach a 
consensus for each scenario through discussion and negotiation—two skills at which 


contracting personnel excel. 


Once the scenarios are identified and the corresponding weights are determined, 
they must then be integrated into the Source Selection Support System. The system uses 
an expert system to do so. Note that the decision tree (including weights) graphically 
represents the collective knowledge of a group of contracting experts. The function of 
the expert system is to transform the tacit knowledge possessed by those experts 
(captured in the decision tree) into explicit knowledge that benefits all contracting 
personnel, including those not nearly as experienced. Thus, the overall system model 
combines information from the proposals and information retrieved from the data 
warehouse with the output of the expert system in order to determine the overall 


acceptance score. 


c. DATA MANAGEMENT 


The data management component is comprised of the data warehouse and built-in 


data mining capability. 


1. Data Warehouse 


The data warehouse for the Source Selection Support System will store 
information required to perform the calculations needed to evaluate the alternatives in 
accordance with the model structure. Recall that certain variables (price, delivery date, 
watranty) are entered via direct data entry once proposals are received. The remaining 
variables (disadvantaged business status, HUB Zone status, customer satisfaction, report 
of discrepancy, an on-time delivery history) require an evaluation of a contractor’s past 
performance information. As such, the data warehouse will store information pertinent to 
the relevant past performance variables for each contractor. At a minimum, the data 
warehouse must include the following data for each contractor in order to produce the 


information necessary to fully apply the model: 


46 


e Disadvantaged business status (Yes/No) 
e HUB Zone status (Yes/No) 
For every contract awarded to and completed by this contractor: 
e Report of Discrepancy record (Yes/No) 
e On-Time Delivery record (Yes/No) 
e Customer Satisfaction Survey Information (as scored by each customer) 


Prior to the initial deployment of the system, the data warehouse must be 
populated with past contract data in order to establish a past performance baseline. 
Logically there will be limits to the amount of data that will be entered due to the large 
amount of data that exists. Rather, the amount of past performance data necessary to 
establish the baseline should be sufficient to form a reasonable representation of what the 
baseline would be if all data were entered. For example, entering past performance data 
from the past five years may be enough to establish a baseline that would mirror the 
baseline if all data had been entered. Inputting the entire history of data is not feasible 
due to the time and money required. Additionally, data may not be as readily available 
for older contracts and even if it is, the older the data, the less its relevance. That is, a 
contractor’s poor performance 25 years ago becomes less relevant if the same 
contractor’s performance over the last five years is stellar. Once the database is current, 
new contractors will be added as they appear and new contracts will be added as they are 
awarded. The data warehouse will be structured in a manner similar to the entity- 


relationship diagram depicted in Figure 7. 
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Figure 7. Data Warehouse entity-relationship diagram 


The Contractor table records the data the model requires for each contractor. 
Since each contractor has a unique cage code, this serves as the primary key. The 
contractor name is also recorded for descriptive and verification purposes. The 
“Disadvantaged” and “HUB Zone” attributes have values of either “Yes” or “No” for 
each contractor. Each contractor may have won multiple contracts, hence the one-to- 
many relationship. The Contract table records relevant data for each contract. The 
primary key is Contract Number (another unique identifier) while “Delivered On Time” 
and “ROD Submitted” are Yes/No attributes. Finally, the survey table records customer 
response data on ten questions from a standardized Likert survey distributed after 
contract completion. Since one contractor may serve more than one customer on the 


same contract (i.e., multiple end users), it is a one-to-many relationship. 


Data quality and integrity is maintained through the near instantaneous saving of 
the contract to the database once the decision is reached, assuming the contracting officer 
concurs with the recommendation. Because the data warehouse is integrated into the 
system, the output of the model (i.e., the recommended contractor) can be easily saved to 
the database without adding much additional data. The main challenge will be to keep 
the database updated with subsequent data after the contract has been awarded. It is easy 


to save the contract award when the contracting officer is already looking at it on his 
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screen, but to log back in to record delivery date, customer satisfaction data, etc. is 
another story. To address this problem, the Source Selection Support System will have a 


feature that lists all contracts with incomplete data when prompted by the user. 


Data processing is required for certain variables. That is, some variables do not 
get their values directly from a particular attribute in a table in the data warehouse. The 
data must first be processed into a new form. For example, the model requires a 
contractor’s customer satisfaction score as an input. Yet there is no attribute in any table 
that provides this information. That is, the data warehouse only records numerical 
responses to individual questions for each survey that is completed. This data must be 
processed in order for the model to accept it. As indicated in the model management 
section, the average score for each survey must be calculated based on the responses to 
each individual question. From there, the average of all survey averages for a contractor 


must be computed to get the information in its proper form. 


The data administration will be based on server administration and database 
standard operating procedures. For example, security will be maintained through 
standard authentication and authorization practices while back-up procedures will include 
regular back-ups kept for a designated period of time at multiple locations to minimize 


the risk of destruction and/or failure. 
2. Data Mining 


Data mining capability will be embedded in the system to serve as a feedback 
enabler. Data mining is “a process that uses a variety of data analysis tools to discover 
patterns and relationships in data that may be used to make valid predictions.”2> 
Essentially, data mining serves to identify patterns and relationships in data, which can 


then be used by managers in decision making. 


Within the context of the Source Selection Support System, data mining can be 


employed to validate the model and update it as needed. For example, recall that one of 


25 Two Crows Corporation. “Introduction to Data Mining and Knowledge Discovery.” Third Edition. 
2005. 
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the initial steps to implement the system is to establish a set of variable weights for each 
possible scenario a contracting officer is likely to encounter. After these weight sets are 
input to the system, contracting officers use them to execute the model. This 
arrangement works well, assuming that the contractor ultimately recommended by the 
contract performs well. If the contractor does not perform well, however, it might be an 
indication that the variable weight mix for the scenario under which the contractor was 
awarded the contract is not optimal. That is, the model may have selected a poor 
alternative and awarded the contract to a contractor who did not offer the overall true best 
value to the government. Alternatively, it may also indicate nothing of consequence. 
Perhaps it was a simple isolated incident, which would not contradict the validity of the 


model. Without further analysis, the true indication cannot be determined. 


Data mining can be used to determine whether patterns of failure are occurring for 
specific variable weight distribution plans. For example, for a particular scenario, 
delivery date score may carry a weight of 15 percent. Subsequent data shows that 
contractors are failing to meet promised delivery dates on a regular basis under this 
scenario. Thus, it may be necessary to modify the weight distribution plan in order to 
increase the weight for another variable such as price at the expense of the delivery date 
score variable, since the delivery date score variable weight is not producing the desired 
effect anyway. Thus, data mining closes the loop in the process by providing feedback to 


the model, as shown in Figure 8. 


50 





Apply 


ame 


Model Award 
Updated Contract 
Data = Work 
Performed 


XN. A 


Recorded 











Figure 8. System feedback loop 


D. USER INTERFACE 


The discussion of the user interface will encompass two broad areas: the manner 
in which users access the system, and the navigation schema once users are inside the 


system. 
1. System Access Using a Virtualization Environment 


One possible disadvantage of implementing the Source Selection Support System 
is the procurement cost associated with the various commercially available decision 
technologies. As the proposed system is intended to be an individual tool that can be 
accessed by acquisition personnel via their desktop computers, the license costs for the 
software packages that serve as the system components could be _ prohibitive. 
Accordingly, a cost effective solution to this constraint would be to allow personnel 
access to the system from their individual workstations without having to install the 
system on each of those workstations. Creating a virtualization environment will do just 
that. 
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Within the context of information technology, virtualization “is a technique for 
hiding the physical characteristics of computing resources from the way in which other 
systems, applications, or end-users interact with those resources.”2° Essentially, 
virtualization allows multiple individual workstations to access the same resource housed 
by a single physical resource. This virtual relationship is shown in Figure 9. Within the 
context of the Source Selection Support System, the system can be installed on a central 
server and accessed by multiple workstations with no physical connection or workstation 


specific software necessary. 

















Figure 9. Virtualization environment for the Source Selection Support System 


26 Andi Mann, “The Pros and Cons of Virtualization.” www.btquarterly.com. Last accessed 12 Feb. 
2008. 
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Virtualization offers multiple advantages. As discussed, there are significant cost 
savings to be realized in the form of reduced software license fees and system 
administration costs. In addition, system maintenance and upgrades are accomplished 
more easily with virtualization. That is, upgrades need only be installed and system 
maintenance need only be performed on the hardware actually hosting the system. 
Corresponding upgrades and maintenance are not necessary on the virtual machines 
accessing the application. Other benefits from virtualization can include better security, 
reduced downtime, increased ability to achieve service levels, the capability to 
accommodate legacy systems on new hardware without major upgrades, and better 


conduciveness to location and staff mobility issues.’ 


Another option for configuring the virtualization environment for the Source 
Selection Support System is to establish a single physical server capable of hosting 
multiple virtual servers. Within the context of DSS, the major benefit of this 
configuration is that it allows for decision support systems which require unique 
configurations to have dedicated virtual hardware as opposed to competing for system 
resources on a shared physical server. Furthermore, this strategy allows virtual servers 
utilizing different operating systems to function without interfering with each other. 
Finally, this configuration enables a data center of smaller size, with corresponding 


savings in server cooling costs. 
2. Navigation Schema 


The user interface provides the portal through which the user accesses the other 
components of the system. Specifically, the interface allows users to input information 
used by the model component, save/retrieve data to/from the data warehouse, and access 
the knowledge captured in the expert system. All of these tasks can be grouped into two 


main system functions: evaluating proposals and database management. The user 





27 Andi Mann, “The Pros and Cons of Virtualization.” www.btquarterly.com. Last accessed 12 Feb. 
2008. 
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navigates through the system via point-and-click. And since the scope of the Source 
Selection Support System is narrow with sequential steps, there is little chance of a user 


getting “lost”. 


If the user wishes to evaluate a set of proposals, he selects this option from the 
main menu. The next step is to calculate the variable weights. In determining the 
weights for the variables, the user will be prompted to answer a series of multiple choice 
questions which will ultimately determine the right mix based on the circumstances 
surrounding the procurement and the business rules that are built into the expert system. 
After the weights are determined, the user searches the data warehouse for the contractors 
from who proposals were accepted. If the contractors are already in the data warehouse, 
the corresponding data for those contractors is retrieved. If a contractor is new and has 
not yet been recorded in the data warehouse, the user will be prompted to do so at that 
time. The user then enters the remaining required data directly after reviewing each 
contractor’s proposal. For each proposal, the user will have to input the price, delivery 


date, and warranty length. 


If the user wishes to perform any database management tasks, he selects this 
option from the main menu. The user has access to all three tables of the database and 
can insert or modify records as necessary. That is, with respect to the Contractor table, 
the user will be able to add new contractors or modify a contractor’s status 
(disadvantaged business and/or HUB Zone). The user will also be able to update records 
in the Contract table. Although the system records a new contract once the user accepts 
the recommendation (assuming the recommendation is accepted), the user will still need 
to update the contract record with on-time delivery and ROD data. A user may also need 
to insert a new contract in the table in the event he does not accept the recommendation 
of the decision support system, thus this capability will be included as well. Finally, the 
user will be able to record customer survey data to the Survey table. Users will not be 
permitted to delete records from any table in order to prevent accidental deletion of 
relevant data. The navigation schema is shown in Figure 10. There are branches from 


the main menu for the two primary activities. 
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The primary users of the Source Selection Support System are federal government 


contracting personnel. These personnel may vary in experience and knowledge, as both 


seasoned contracting officers and brand new contracting support staff are potential users. 


A possible secondary group of users may be the customers on whose behalf the contracts 


are awarded. That is, if the system is upgraded to include a web-enabled capability, 


customers will be able to input their own survey data as opposed to contracting personnel 


performing that task. 


JD 


The preceding discussion of the proposed system discussed the structure of the 
model in largely conceptual terms, without identifying any particular decision 
technologies to be used. As such, the next step is to take the model and research what 
commercially available decision technologies most appropriately align with the 
objectives of the system. After determining which decision technologies to incorporate 
into the overall system, the individual elements of the system (weight-based ranking 


model, expert system, data warehouse) can be constructed. 
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IV. SYSTEM PROTOTYPE 


The Source Selection Support System links commercially-available decision 
support software systems with a database using a custom designed user interface. 
Specifically, the prototype uses Infoharvest Corporation’s Criterium Decision Plus for the 
weight-based ranking system, Informavore Corporation’s Firefly Designer for the expert 
system, and Microsoft Access for the database. These systems interact in the manner 
depicted in Figure 11. The proceeding scenario will be used to illustrate how these 


individual software systems function to support the overall system. 
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Figure 11. System prototype software interrelationships 
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A. SCENARIO 


USS Neversail, a guided-missile destroyer homeported in San Diego, CA requires 
24 new tactical combat vests for boarding team members. The ship deploys in four 
weeks and these are mission critical items. And since team members may depend on 
these vests to save their lives, a strong warranty is highly desirable. The total cost is 
expected to be between $30,000 and $35,000. Since the job will exceed the micro- 
purchase threshold of $2,500, the ship submits the requirement to the local contracting 
activity at the Fleet Industrial Supply Center. The contracting officer has received the 
request, issued a solicitation, and received the three proposals required under simplified 
acquisition procedures. All proposals meet technical requirements. Additionally, the 
contracting officer expects to award the contract on January 14, 2008. As expected, the 
proposals are slightly different when it comes to price, but the contracting officer is not 
convinced that the lowest price is the best decision for the government. He examines the 


proposals more closely and collects the data shown in Table 12. 





Davis Army Supply International Security Shipboard Solutions 











Delivery Date 6/5/2008 5/11/2008 6/15/2008 
Price $32,500 $31,300 $30,475 
Warranty 0 42 48 




















Table 12. Proposal data for USS Neversail scenario 


Davis Army Supply is a disadvantaged business, and therefore qualifies for a ten 
percent price adjustment to its initial bid. The other two contractors do not qualify for 
any price adjustments. The contracting officer logs on to the Source Selection Support 


System in order to help him arrive at an award decision. 
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B. EXPERT SYSTEM 


Following the system navigation schema in Figure 10, the first step is to 
determine the appropriate weights for the independent variables given the characteristics 
of the scenario. This is accomplished through the Firefly expert system. Figure 12 


shows the expert system structure. 
The user has four alternatives for price importance: 
e Less than $25,000 
e Between $25,000 and $50,000 
e Between $51,000 and $75,000 
e Between $75,000 and $100,000 
Three alternatives for delivery date importance: 
e Mission critical 
e Delivery requested as soon as possible but not mission critical 
e Delivery at required delivery date sufficient 
And three alternatives for warranty importance: 
e Warranty very important 
e Warranty somewhat important 


e Warranty not important 
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Weights For Price ... 
Figure 12. Source Selection Support System expert system component structure 


For the purposes of this prototype, the remaining three variables (on-time delivery 
score, ROD score, and customer satisfaction score) represent equal portions of the 
remaining weight percentage available. For example, if the combined weight of the 
price, delivery date, and warranty variables is 70 percent, then the weight for each of the 
remaining three variables is 10 percent [(100-70)/30]. Firefly includes a feature that 
allows the user to proceed through each of the three multiple-choice variables in order to 
arrive at the pre-determined weight distribution. The user is prompted to answer 


questions similar to the one displayed in Figure 13. 
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Within what range will the contract price 
likely fall? 

© Less than $25,000 

® $25,000 - $50,000 

© $50,000 - $75,000 

© $75,000 to $100,000 











Figure 13. Sample Firefly question 


As shown in Figure 14, after the user has answered all required questions, Firefly 


provides a results screen that displays the resultant variable weights. 





Price weight is: 55.00 %. 

Delivery Date weight is: 15.00 %. 
Warranty weight is: 10.00 %, 

Customer Satisfaction weight is: 6.67 %. 
ROD history weight is: 6.67 %., 

Ontime Delivery Percentage weight is: 
6.67 %. 











Figure 14. Firefly results screen 


Now that the variable weights have been determined, it is time to calculate the 


scores for each variable. 
C. DATA WAREHOUSE 


The system prototype uses a Microsoft Access database as the data warehouse. 
Recall that the system retrieves scores for the On-Time Delivery, ROD, and Customer 
Satisfaction variables from the data warehouse. The Access schema is shown in Figure 
1, 
61 


=; Relationships 


CONTRACTOR CONTRACT SURVEY 


Cage_Code Contract_Number 

Name Delivered_On_Time 

Disadvantaged 5 | ay) |ROD_Submitted 
Cage_Code 


— Contract_Number 





Figure 15. Microsoft Access schema 


Queries are used to convert the data recorded in the tables into the scores needed 
by the weight-based ranking model. For example, if a contracted item was delivered late, 
a “No” is recorded in the Delivered_On_Time attribute of the Contract table. A query 
retrieves all contract records for a particular contractor and calculates the overall On- 
Time Delivery score for that contractor through various structured query language 
statements. Once the system has retrieved the On-Time Delivery, ROD, and Customer 
Satisfaction scores, it stores them in the weight-based ranking model built in Decision 


Plus. 
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The data warehouse provided the data shown in Table 13 for the three contractors 


in this scenario. 











Davis Army International Shipboard 

Supply Security Solutions 
ROD Score 94 % 90 % 48 % 
Customer Satisfaction Score 94 % 100 % 48 % 
On-Time Delivery Score 79 % 78 % 79 % 




















Table 13. Additional Information From Data Warehouse 


D. WEIGHT-BASED RANKING MODEL 


The first step in creating a model in Decision Plus is to create a goal hierarchy. 
For the Source Selection Support System, the ultimate goal is to select a contractor to 
recommend to the contracting officer for contract award. Thus, “Select a Contractor” is 
the Goal level in the hierarchy. The variables that play a role in determining the outcome 
of the goal level are listed in the next level of the hierarchy, Level 2. Finally, since each 
contractor will be scored against these variables, they are all modeled in the hierarchy as 
well in the form of alternatives. The completed hierarchy for the Source Selection 


Support System is shown in Figure 16. 
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Figure 16. Decision Plus hierarchy 


After modeling the hierarchy, the next step is to transfer the weights calculated in 
Firefly into the Decision Plus module. Recall from Figure 13 that the variable weights 


for this scenario are as follows: 
e Cost: 55 percent 
e Delivery Date: 15 percent 
e Warranty: 10 percent 
e Customer Satisfaction: 6.67 percent 
e ROD History: 6.67 percent 


e On Time Delivery Percentage: 6.67 percent 


64 


The Decision Plus screen used to enter the variable weights is shown in Figure 17. 
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Figure 17. Decision Plus variable weight entry screen 


Decision Plus offers three ways in which to enter variable weights. The first 
method is to directly enter the values (15, 55, 10, etc.) after selecting the appropriate units 
and range of acceptable values. Alternatively, the user can change weight values by 
sliding the corresponding bar charts left (to decrease) or right (to increase). Finally, the 
user can choose a descriptive term from a drop down menu that best describes the relative 
importance of that variable. These terms include critical, very important, important, 
unimportant, and trivial. Each term has a default numerical weight associated with it 


(100, 75, 50, 25, 0 respectively). The Source Selection Support System will use the first 
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method, since the calculated weight percentages are too precise for either of the two 


remaining options to be feasible in certain scenarios. 


Once the hierarchy is constructed and the variable weights are imported, the next 
step is to enter each contractor’s scores for the variables. The user must enter the scores 
for cost, delivery date, and warranty while the data warehouse will automatically supply 
the scores for the remaining three variables. The input screen for the direct entry 
variables is shown in Figure 18. Note that this screen must be completed for each 


contractor. Figure 18 displays data for Contractor A (Davis Army Supply). 





ie) Rate Alternatives 


Options Help 


Delivery Date Percent 
Price Percent 
Warranty Percent 
ROD History Percent 
Customer Satisfaction Percent 
Ontime Delivery History Percent 


Restore 





Figure 18. Variable Scores for Contractor A (Davis Army Supply) 


Once the hierarchy is constructed, the variable weights are imported, and the 
variable scores for each contractor are entered, the model is ready to be run. The default 


results screen is shown in Figure 19. 
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Figure 19. Default Results Screen 


In this scenario, contractor B (International Security) is the recommended 
contractor with an overall acceptance score of 92.8 percent. Recall that International 
Security’s proposed price was not the lowest of the three proposals. In fact, it was $825 
(almost three percent) higher than the lowest proposed price—that of Shipboard 
Solutions. Thus, although International Security had a lower score for the price variable, 
it was still able to earn the recommendation because it outperformed the other two 
contractors with respect to the non-price variables. The Decision Plus results match those 
the user would obtain had he manually calculated the variable scores for each contractor 
and applied the corresponding weights. Figure 20 shows the individual variable results 


for each contractor. 





Davis Army Suppl Shipboard Solutions 
Variable Variable Variable | Weighted | Variable | VWveighted | Variable | Weighted 
Weight Score Score Score Score Score Score 


Price* 55.00% 100.00% 95.61% 


Delivery Date** 15.00% 79.31% ; } : 70.69% 
ane 10.00% 0.00% I . : 100.00% 

6.67% 94.00% : ; i 48.00% 

Ontime Delivery 6.67% 94.00% ; f f 43.00% 
Customer Satisfaction 6.67 % 79.00% 79.00% 79.00% 


Overall Acceptance Score 84.70% i — 92.76% a 84.97% 





Figure 20. Individual Variable Results For Each contractor 
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Additionally, Decision Plus allows the user to view the results in stacked bar 
graph format, as shown in Figure 21. In this view, it is easier for the user to see why a 
particular contractor is recommended over the others. In this scenario, although 
International Security (Contractor B) earned the least score for the price variable, its first 


place delivery date score helped to compensate for that shortcoming. 



































Contributions to Select a Contractor from Level:Level 2 
1.0 1.0 
0.9 5 0.9 
08 08 
0.7 0.7 
Bl Price 
0.6 0.6 [i Delivery Date 
0.5 + 05 BB Warranty 
LJ Ontime Delivery History 
0.4 4 y 0.4 BB ROD History 
0.3 0.3 [fj Customer Satisfaction 
0.2 5 0.2 
0.1 + 0.1 
0.0 0.0 
Contractor B Contractor C Contractor A 
Figure 21. Stacked Bar Graph Results Screen 


Finally, Decision Plus has a sensitivity analysis feature. In order to conduct a 
sensitivity analysis, the user chooses a particular variable. Decision plus presents the 
results of the analysis for this variable in the form of a line chart with a vertical line that 
the user can slide horizontally, as shown in Figure 21. There is a different line on the 
chart for each contractor and the points at which the lines intersect represent points where 
the recommended solution would change. That is, the x-axis of the line chart represents 
the weight for that variable. The user can slide the vertical line left or right to modify the 
variable weight, shown in the lower right corner of the screen. The points of intersection 
represent what the variable weight would have to be in order for the final solution to 
change. For example, Figure 22 confirms that Contractor B, International Security, is the 
recommended contractor given a weight of 10 percent for the warranty variable (see 


lower right of screen). 
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Figure 22. Initial Sensitivity Analysis Screen For Warranty Variable 


Note that the lines for International Security and Shipboard Solutions (contractor 
C) intersect to the right of the vertical line. This represents the warranty weight 
percentage that would result in Shipboard Solutions becoming the recommended 
contractor. As shown in Figure 23, if the weight for the warranty variable is increased to 
exceed approximately 45 percent (refer to the lower right corner of Figure 23), the 


solution changes. 
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Figure 23. Revised Sensitivity Analysis Screen For Warranty Variable 
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Due to the programming required and the author’s lack of programming expertise, 
a user interface was not constructed. However, the preceding prototypes of the individual 
components are sufficient to provide a sense of how the entire system will function once 
fully developed. As stated in the Introduction, a fully functional system is not within the 


scope of this thesis, but serves as an exciting opportunity for further development. 
E. USER ACCEPTANCE 


As with any new system, the user acceptance of the Source Selection Support 
System will likely not be immediate. The contracting community is a highly specialized 
workforce, trained to exercise judgment influenced by experience, education, and 
training. They are empowered with considerable autonomy to make decisions that 
commit taxpayer dollars. As such, any system that perceivably decreases the level of 
autonomy with which they are entrusted will likely meet with resistance. Upon 
introduction of the system, a popular refrain within the contracting community may be 
that an automated system cannot duplicate the human thought process that is used to 
select sources for government work. This perception can be countered by successfully 
conveying the message that the model is a product of the human thought process involved 
in the source selection decision and not simply an information system haphazardly 
developed to automate a formerly human process. Furthermore, the fact that it is a static 
model, updated as necessary in response to the output of the embedded data mining 
capabilities, must be communicated to acquisition personnel in order to reinforce the fact 
that the system is a reflection of human thought and reason as opposed to a replacement 
for those activities. That is, the Source Selection Support System is not designed to 
replace the human decision maker. Rather, since it is a decision support system, it serves 


as a tool to assist the decision maker in arriving at a decision. 


Once the intent is clearly communicated to the contracting community, those 
personnel will be able to see several appealing characteristics of the system. These 
include a time-saving aspect, as contracting personnel will not have to spend as much 
time performing tradeoff analyses. Additionally, the system provides contracting officers 


with a more defensible position in awarding a contract to a particular contractor should 
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there be a protest. From a command standpoint, contracting personnel will be able to 
increase contract award throughput, thereby reducing customer response time as well. 
Finally, perhaps the most appealing benefit is the knowledge that the contractor offering 


the best value to the government was awarded the contract. 
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Vv. CONCLUSION 


A. SUMMARY DISCUSSION 


Under current procedures, federal government contracting personnel are 
empowered to evaluate competing contractor proposals and exercise personal, 
professional judgment in determining which contractor is awarded the contract. This 
approach is highly subjective in that contracting personnel need to perform tradeoff 
analyses based on personal experience and acquired knowledge. As experience and 
knowledge levels can vary greatly from person to person, it is highly possible that two 
contracting officers, when evaluating the same set of proposals, could arrive at a different 
conclusion. And as only one contractor can offer the true best value to the government, 
the contracting officer who would award the contract to a different contractor would 
clearly be making an error in judgment. There is little consistency in the proposal 
evaluation process and that lack of consistency is contributing to improper selection of 


sources. 


There are multiple commercially available decision technologies contracting 
personnel can employ to establish not just consistency in the evaluation process, but 
validity as well. Through the integration of these technologies, contracting personnel can 
leverage their capabilities into forming a useful source selection tool. This is what the 


Source Selection Support System seeks to accomplish. 


The Source Selection Support System combines a multi-criteria decision analysis 
system with an expert system and a data warehouse to form an objective evaluation 
model. The system is structured around a weight-based ranking model and links the 
various individual software components through a custom designed user interface. 
Furthermore, the ideal system is distributed via a virtualization environment, where 


multiple users virtually connect to a single system installed on a central server platform. 
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Ultimately, the Source Selection Support System is designed to provide the 
contracting officer with as to which contractor offers the “best value” proposal in a 
competitive solicitation by weighing multiple variables such as price, delivery terms, and 


past performance measures. 
B. RECOMMENDATIONS FOR FURTHER RESEARCH 


There are two primary areas where further research is recommended. First, the 
prototype currently does not represent the complete vision of the system. While 
individual components (weight-based ranking model, expert system, and data warehouse) 
have been constructed, the user interface and data mining functionalities have yet to be 
developed. In addition, once the prototype is fully functional, experimentation in a 


virtualization environment is recommended prior to system deployment. 


A second area of research that may be explored centers around creating an 
implementation strategy for a test deployment of the system. This research will involve 
issues such as feasibility studies, hardware and software procurement plans, data 


warehouse population strategies, and management of organizational change. 


A final possible area of research involves the application of integrated decision 
technology to areas within the acquisition domain other than simplified acquisition 
procedures. Where simplified acquisition procedures represents a structured problem 
well suited to the proposed model, there are numerous facets within the acquisition arena 
that are not as structured. For example, the problem of determining what to buy is 
considerably less structured than the problem of determining from whom to buy a 
particular item. As such, it would be useful to explore how integrated decision 


technologies can assist with decisions such as these as well. 
C. FINAL THOUGHTS 


Decision support systems are typically single scope applications that focus on one 
type of application domain. While this is acceptable for many decision environments, it 
does not fully harness the capability of decision support systems. This thesis focused on 


how an integrated decision technology environment can be employed to assist contracting 
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personnel in determining which contractor offers the best value as compared to other 
contractors competing for the same contract. While the proposed system combines the 
concepts of multi-criteria decision analysis, expert systems, and data mining, it also 
serves to indicate the general potential of integrating multiple decision technologies to 
spawn decision support system generators for complex decision-making problems. That 
is, decision technologies such as agent-based simulation, optimization, social network 
analysis, and other application domains can potentially be integrated to address complex 
problems that previously could not be adequately addressed by a single-scope decision 
technology. Decision technology integration, as proven in this thesis, opens new doors to 
solving problems previously considered too complex for standard decision support 


systems. 
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